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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). (n no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to communication(s) filed on 14 June 2005 . 
2a)[x] This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) M Claim(s) 1-16.18-25 and 28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 1-16. 18-25. 28 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . This office action is responsive to the following communications: Application, 
filed 03/27/2001 ; Amendment A, filed 06/30/2003; Amendment B, filed 1 1/10/2003; 
Amendment C, filed 12/15/2003; Amendment D, filed 05/03/2004; Amendment, filed 
12/20/2004. 

2. Claims 1-16, 18-25 and 28 are pending in this application. Claims 1, 13 and 25 
are independent claims. Claim 28 has been added. 

3. The present title of this application is "Augmented Reality Guided Instrument 
Positioning with Modulated Guiding Graphics" (as originally filed). 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-16, 18-25 and 28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Chen et al., U.S. Patent Number 5,765,561 , class 600/407 in view of 
Fukunaga, U.S. Patent Number 6,346,940 B1, "Virtualized Endoscope System", class 
345/427, 02/2002, filed 02/1998. 



Regarding claim 1, Chen discloses a method for augmented reality guided 
instrument positioning, comprising the steps of: displaying a real view of an environment 
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(col. 9, lines 5-7); marking the preferred path with a graphics guide (col. 8, lines 26-30); 
augmenting the real view with a rendering of the graphics guide (col. 10, lines 1-12) 
such that at least one portion of the graphics guide is transparent with respect to other 
portions of the graphics guide to provide a substantially unobstructed view through the 
at least one portion of the graphics guide to at least a portion of the instrument (col. 10, 
lines 15-29); aligning the instrument with the graphics guide so that the instrument 
appears in a same location as the graphics guide in the augmented view, and when 
properly aligned, at least a portion of the instrument is visible through the at least one 
transparent portion of the graphics guide (col. 10, lines 29-33); and inserting the 
instrument in the graphics guide. 

However, it is noted that Chen fails to disclose an instrument included in the real 
view of the environment, determining a preferred path for positioning of said instrument, 
and inserting the instrument in the graphics guide. 

Fukunaga discloses a real electronic endoscope system to prepare virtual 
images containing endoscopic guide data, col. 5, lines 30-39, which Examiner interprets 
as including an instrument in a real view of the environment. Fukunaga further 
discloses endoscopic guide data, which Examiner interprets as a preferred path of the 
instrument and insertion of the instrument in the graphics guide, col. 8, lines 1-30. 

It would have been obvious to one of ordinary skill in the aft at the time of the 
invention of Chen to include in the real image the instrument included in the 
environment for the medical procedure as disclosed in Fukunaga to enable an operator 
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to quickly and easily obtain guiding images for real instruments in preoperative 
simulation and simulate control of the actual instrument. 

Regarding claim 2, Chen discloses rendering includes a modulation of the 
graphics guide's transparency along the length of the graphics guide, so that a plurality 
of portions of the graphics guide with respect to other portions of the graphics guide 
along the length of the graphics guide to provide a substantially unobstructed view 
through the plurality of transparent portion of the graphics guide; and inserting the 
instrument in the graphics guide (col. 10, lines 15-33, fading away, which Examiner 
interprets as modulating, the virtual image, i.e. graphic guide or planning marker) 

Regarding claim 3, Chen discloses a transparent guide marker (col. 10, lines 15- 

33) 

However, it is noted that Chen fails to disclose varying the transparency of each 
of a plurality of portions of the graphics guide during at least one predefined time 
interval to provide a substantially unobstructed view through each of the plurality of 
portion to at least a portion of the instrument. 

Fukunaga discloses varying the transparency of each of a plurality of portions of 
the graphics guide during at least one predefined time interval to provide a substantially 
unobstructed view through each of the plurality of portion to at least a portion of the 
instrument (col. 7, lines 59-64) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include in the transparency of the graphics marker disclosed in Chen the 
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varying of the portions to allow a user to view hidden portions as disclosed in Fukunaga, 
col. 11, lines 13-19. 

Regarding claim 4, Chen discloses a transparent guide marker (col. 10, lines 15- 

33) 

However, it is noted that Chen fails to disclose varying the transparency of each 
of a plurality of portions of the graphics guide during at least one predefined time 
interval to provide a substantially unobstructed view through each of the plurality of 
portion to at least a portion of the instrument. 

Fukunaga discloses varying the transparency of each of a plurality of portions of 
the graphics guide during at least one predefined time interval to provide a substantially 
unobstructed view through each of the plurality of portion to at least a portion of the 
instrument (col. 7, lines 59-64) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include in the transparency of the graphics marker disclosed in Chen the 
varying of the portions to allow a user to view hidden portions as disclosed in Fukunaga, 
col. 11, lines 13-19. 

Regarding claim 5, Fukunaga discloses wherein plurality of portions is 
consecutive, (col. 8, lines 5-10) 

Regarding claim 6, Chen discloses a transparent guide marker (col. 10, lines 15- 

33) 

However, it is noted that Chen fails to disclose varying the transparency of each 
of a plurality of portions of the graphics guide during at least one predefined time 
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interval to provide a substantially unobstructed view through each of the plurality of 
portion to at least a portion of the instrument. 

Fukunaga discloses varying the transparency of each of a plurality of portions of 
the graphics guide during at least one predefined time interval to provide a substantially 
unobstructed view through each of the plurality of portion to at least a portion of the 
instrument (col. 7, lines 59-64) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include in the transparency of the graphics marker disclosed in Chen the 
varying of the portions to allow a user to view hidden portions as disclosed in Fukunaga, 
col. 11, lines 13-19. 

Regarding claim 7, Chen discloses constructing the graphics guide as a line, and 
rendering step comprises the step of modulating the transparency of the line with 
respect to time so that the line repeatedly fades in and out of view to provide a 
substantially unobstructed view . . . (col. 10, lines 15-33) 

Regarding claim 8, Chen discloses constructing the graphics guide as a line, and 
rendering step comprises the step of modulating the transparency of portions of the line 
so that at least a portion of the instrument is substantially unobstructed . . . (col. 8, lines 
26-30 and col. 10, lines 26-33) 

Regarding claim 9, Chen discloses rendering step comprises the step of 
modulating the transparency of portions of the line with respect to time and space so 
that at least a portion of the instrument is substantially unobstructed . . . during pre- 
defined time intervals (col. 10, lines 15-33) 
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Regarding claim 10, Chen discloses constructing the graphics guide as a cylinder 
. . . rendering step comprises the step of modulating the transparency of the cylinder 
with respect to time so that the cylinder repeatedly fades in and out of view . . . (virtual 
planning marker consist of any geometric form and a path, col. 8, lines 26-30, which 
Examiner interprets as inclusive of a cylinder) 

Regarding claim 1 1 , Chen discloses use of any geometric form for virtual 
planning markers, col. 8, lines 26-19 and further transparency of the virtual planning 
markers, col. 10, lines 15-22) 

However, it is noted that Chen fails to disclose varying the transparency of each 
of a plurality of portions of the graphics guide during at least one predefined time 
interval to provide a substantially unobstructed view through each of the plurality of 
portion to at least a portion of the instrument. 

Fukunaga discloses varying the transparency of each of a plurality of portions of 
the graphics guide during at least one predefined time interval to provide a substantially 
unobstructed view through each of the plurality of portion to at least a portion of the 
instrument (col. 7, lines 59-64) and further discloses modulating the transparency of 
portions of the cylinder so that at least a portion of the instrument is substantially 
unobstructed . . . (semitransparent image so that portions of the image can be viewed 
unobstructed, col. 1 1 , lines 44) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include in the transparency of the graphics marker disclosed in Chen the 
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varying of the portions to allow a user to view hidden portions as disclosed in Fukunaga, 
col. 11, lines 13-19. 

Regarding claim 12, Chen discloses rendering step comprises the step of 
modulating the transparency of portions of the cylinder with respect to time and space 
so that at least a portion of the instrument is substantially unobstructed . . . during pre- 
defined time intervals (col. 10, lines 20-29) 

Regarding claims 13-16 and 18-24, they are rejected based upon similar rational 
as above claims 1-11 respectively. Chen further discloses a video camera providing a 
real view of an environment, figure 1 , element 45 and 45'. 

Regarding claim 25, it is rejected based upon similar rational as above claim 2. 

Regarding claim 28, Chen discloses the transparency of the graphics guide is 
modulated with respect to time (col. 10, lines 26-29, fading away some or all of the 
virtual image, which Examiner interprets as modulating with respect to time) 

Response to Arguments 

6. Applicant's arguments filed 06/14/2005 have been fully considered but they are 
not persuasive. 

7. In response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies 
(i.e., exposing a mutual spatial relationship between the real instrument and the virtual 
guide to assist in alignment of the real instrument, page 10, remarks) are not recited in 
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the rejected claim(s). Although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

Applicant argues that Chen discloses purely virtual images for guiding surgical 
procedures and that the present invention combines images of virtual guide and real 
instrument to assist a physician in guiding the instrument. Chen discloses mixing real 
images and virtual images on display to facilitate a medical procedure, col. 10, lines 1- 
15, having virtual planning marker rendered with the structure being transparent or 
semi-transparent, col. 10, lines 16-33. Applicant argues that the present invention 
combines images of virtual guide and real instrument without the need of an external 
tracking system. Applicant claim language fails to recite without the need of an external 
tracking system. 

Applicant argues that Fukunaga teaches guiding marker preparation unit forming 
guiding markers and that Fukunaga fails to disclose augmenting the real view with a 
rendering of the graphics guide. It is the position of the Examiner that Chen teaches 
augmenting a real image with a virtual image, col. 10, lines 1-12. Fukunaga discloses a 
real instrument in a real image and guiding markers. Applicant argues that Fukunaga 
fails to disclose overlaying the real view with the virtual map for precise spatial 
alignment. It is the position of the Examiner that Chen discloses overlaying, i.e. mixing, 
a virtual image with a real image. 
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Applicant further argues that Fukunaga fails to disclose at least one portion of the 
graphics guide is transparent with respect to other portions of the graphics guide. 
Fukunaga discloses in figure 18, guide markers Y1, Y2, and Y3, each showing a 
transparent portion with respect to other portions of the graphics guide. Applicant 
argues that the guide markers are not intended to mark exact spatial endoscope 
locations inside the channels. Applicant fails to claim this limitation. 

Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Motilewa A. Good-Johnson whose telephone number is 
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(571 ) 272-7658. The examiner can normally be reached on Monday, Tuesday and 
Thursday 9:00 AM - 6:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone 
number for the organization where this application or proceeding is assigned is 703- 
872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Motilewa A. Good-Johnson 

Examiner 
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